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Abstract 


A  battle  space  contains  information  that  is  dynamic,  unstructured,  uncertain,  and 
temporal.  The  commander  must  use  this  information  to  plan,  analyze,  and  decide  on  the 
“best”  strategy  to  defeat  the  enemy.  Such  information  must  be  context-specific  and 
content-sensitive  to  changes  in  the  battle  space.  Typically,  the  commander  must  integrate 
information  from  various  sources  (terrain,  weather,  resources,  equipment,  personnel,  and 
mission  definition)  to  develop  courses  of  action  and  executable  operation  order.  A  typical 
collection  asset  decision  aid  will  be  useful  to  the  commander  in  planning  for  resource 
management,  courses  of  action  (COA)  analysis,  logistics,  and  sample  profiles  on  the 
order  of  battle  operation.  This  paper  describes  an  analytical  decision  aid  for  supporting 
the  commander’s  courses  of  action  planning  and  analysis  based  on  battle  asset 
information.  The  METT-T  (Mission,  Enemy,  Troops,  Terrain,  and  Time)  paradigm  is  the 
main  knowledge  that  drives  the  Alternative  Courses  of  Action  Display  (ACAD)  software. 
Usability  test  of  the  ACAD  shows  its  utility  to  expert  and  novice  battle  planners  have 
favorable  impression  and  acceptable  trust  metric  scores 


Introduction 

Because  of  technology,  the  amount  of  information  that  military  commanders  and 
staffs  must  process  has  increased  tremendously  since  World  War  II  and  the  amount  of 
time  available  for  decision-making  has  decreased  dramatically.  What  remains  unchanged 
is  the  cognitive  capacity  of  the  human  decision-  maker.  According  to  Wohl  (1981),  the 
growing  complexity  of  battlefield  decision-making,  when  combined  with  increased  data 
input  of  new  sensor  technology  and  reduced  decision  time  can  benefit  from  computerized 
decision-making  aids.  Such  aids  can  structure,  generate,  and  help  the  commander  to 
assess  the  different  hypotheses  and  options  needed  to  conduct  an  effective  tactical 
maneuver.  Decision  aids  are,  ubiquitously  used  today  to  support  the  commander  in 
military  decision-making  process  (MDPS).  The  Alternative  Courses  of  Action  Display 
(ACAD)  software  is  one  of  such  aids 

To  the  commander  and  the  battle  staff,  information  is  required  for  modeling  the 
MDMP.  These  tasks  include,  among  other  things,  planning,  analyzing,  selecting,  and 
executing  alternative  courses  of  action  (DOD,  1997a,  1997b). 

A  typical  MDMP  consists  of  four  essential  steps: 

1.  Mission  analysis.  This  requires  the  commander  to  create  a  prototype  of  the  battle 
situation,  conduct  scenario  analysis,  and  define  the  goal  and  objectives  to  be 
accomplished. 

2.  Course  of  action  (COA)  development.  At  this  stage,  the  commander  identifies 
various  opportunistic  strategies  that  may  or  may  not  result  in  a  mission 
achievement.  These  opportunistic  strategies  are  either  accepted  or  rejected  based 
upon  the  mission  objectives  and  the  potential  risks  associated  with  the  strategy. 

3.  Courses  of  action  analysis.  Here  the  commander  with  his  staff,  provide  an 
integrated  view  of  the  battle  strategy  including  resource  requirements,  avenues  of 
approach  (AA),  and  risks  associated  with  a  particular  strategy.  Usually,  a  COA 


analysis  compares  the  friendly  and  enemy  information  and  uses  it  to  conduct 
trade-off  studies  in  terms  of  risk  and  mission  objectives. 

4.  Decision.  At  the  decision  stage,  the  commander  selects  the  COA  he/she  believes 
to  be  the  most  advantageous. 

In  general,  courses  of  action  modeling  constitute  the  primary  task  of  the  commander  and 
his/her  staff  at  the  strategic  planning  stage  of  MDMP.  A  course  of  action  is  a  multi¬ 
attribute,  multi-criteria  decision-making  problem  because  of  the  many  variables  required 
in  the  modeling  process. 

Within  the  framework  of  a  course  of  action  (COA)  model,  the  commander  must 
estimate  the  necessary  information  about  his/her  force,  as  well  as  estimating  equal  or 
more  information  on  the  enemy  force.  The  results  derived  from  such  efforts  determine, 
in  part,  how  the  battle  is  to  be  fought  as  detailed  in  a  production  order  issued  to  field 
commanders.  The  approval  of  derived  operation  plans  and  operation  orders  (OPLAN  and 
OPORD)  by  the  higher  level  commander  results  in  a  mission  execution  (DOD,  1995). 

Motivation  for  Developing  ACAD 

A  key  difference  between  attrition  warfare  -  the  wearing  down  of  an  enemy  -  and 
maneuver  warfare  -  high  tempo  warfare  -  is  the  method  of  engaging  the  enemy.  Attrition 
warfare  is  the  application  of  friendly  strength  against  an  enemy's  strength.  It  is  typically  a 
"linear"  or  two-dimensional  style  of  fighting  that  is  frequently  indecisive  and  inherently 
costly  in  tenns  of  personnel,  resources,  and  time.  The  concept  of  attrition  warfare  is 
especially  suitable  for  constructive  modeling  and  simulation  as  human  and  equipment 
resources  can  be  aggregated  at  the  atomic  level  of  granularity  to  derive  a  single  measure 
of  performance. 

Courses  of  action  planning  with  “attrition”  criterion  have  been  of  interest  to 
military  planners.  The  current  practice  is  to  use  computer  modeling  and  simulation  to 
support  such  objective.  However,  most  of  the  existing  COA  decision  support  systems 
lack  the  inclusion  of  “intangibles”  and  representation  of  the  commander’s  intent  in  their 
attrition  models. 

At  the  planning  phase  of  the  battle,  estimating  attrition  for  both  the  enemy  and 
friendly  forces  is  a  rather  complex  and  difficult  task  because  of  information  density  to  be 
analyzed.  For  the  most  part,  information  may  be  incomplete,  excessive,  subjective,  and 
multi-dimensionally  heterogeneous.  This  poses  a  significant  problem  to  the  commander 
and  the  battle  staffs  who  are  tasked  to  use  the  available  infonnation  to  determine  the 
“best”  course  of  action  to  defeat  the  enemy.  Defeating  the  enemy,  generically,  translates 
to  combining  the  “best”  available  friendly  resources  to  fight  against  the  enemy  resources. 
In  the  ACAD  model,  we  attrite  the  enemy  through  systematic  and  interactive  instantiation 
of  the  commander’s  intent,  asset  combination,  as  well  as  intangibles  that  reflect  the  battle 
environment  and  personnel  morale.  The  commander’s  intent  is  particularly  important  in 
cases  where  the  situation  that  gives  rise  to  orders  changed.  The  ACAD  model  can 
simulate  such  an  unexpected  scenario  by  allowing  the  user  to  reconfigure  the  necessary 
battle  assets  to  match  the  new  situation. 


Related  Work  on  Military  Decision  Aiding  Tools 


There  many  battle  planning  decision  aids  available  (Ntuen  and  Park,  2003).  The 
following  decision  aids  have  historical  relationships  to  ACAD  development.  They  are: 

FOX-GA:  This  is  a  COA  generation  tool  developed  at  UIUC  by  Schlabach  and  Hayes 
(1998),  an  Army  Intel  officer.  A  genetic  algorithm  model  developed  drives  FOX-GA  to 
mimic  the  state-space  mappings  (enumerative  search)  predetermined  friendly  force 
strength  to  that  of  the  enemy.  The  GA  uses  a  niching  strategy  to  ensure  the  quality  of  a 
selected  COA.  The  major  input  variables  to  FOX-GA  are  avenues  of  approach,  tactical 
assembly  areas,  terrain  objectives,  the  forward  edge  of  the  battlefield  (FEBA),  and  lines 
of  defensive  terrain. 

CORAVEN:  This  is  a  model  developed  for  Intelligence  Collection  Management  (ICM) 
and  analysis  (Jones,  Wilkins,  Bargar,  Sniezek,  and  Asaro,  2000).  The  sub-functions  of 
CORAVEN  include  Requirements  Management,  Mission  Management,  Asset 
Management,  and  Dissemination.  CORAVEN  runs  in  a  multimedia  system  to  support 
intelligence  analyst.  It  uses  Bayesian  belief  network  as  a  modeling  tool.  It  also  has  an 
embedded  prototype  sonification  data  streams  to  enhance  situation  awareness  by 
supporting  alarms  and  warnings  about  false  events. 

OWL:  This  is  a  decision-analytic  COA  wargaming  tool  developed  by  Rockwell  Science 
center  (Uckun,  Tuvi,  Winterbottom,  and  Donohue,  1999)  for  predicting  alternative 
outcomes  of  a  battle  based  on  uncertain  infonnation  available  about  friendly  and  enemy 
forces.  The  infonnation  used  consists  of  mission,  weather,  and  tenain.  OWL  is  designed 
as  a  post-processor  for  FOX-GA.  OWL  executes  the  same  wargame  scenario  in  several 
iterations  each  with  randomly  generated  inputs  derived  from  a  defined  probability 
distribution  function.  The  input  to  OWL  is  modified  combined  obstacles  overlay 
(MCOO)  represented  by  a  graph  structure,  a  set  of  possible  friendly  courses  of  action 
(FCOAs),  and  a  set  of  enemy  courses  of  action  (ECOAs). 

Description  of  The  ACAD  Model 

The  ACAD  model  is  designed  to  implement  analytical  and  behavioral  models  of  a 
battle  scenario.  The  key  outputs  of  ACAD  include  (1)  detennining  the  predictive  battle 
state  based  on  the  RFR  calculation,  and  (2)  graphical  and  textual  portrayal  of  expected 
attrition  index  on  hourly  basis  during  a  selected  battle  mission  time,  and  the  calculation 
of  potential  vulnerability  for  the  friendly  troops.  The  ACAD  model  calculations  consider 
the  following:  (a)  mobility  factors  (b)  personnel  attrition,  (c)  attrition  of  weapons,  (d) 
resupply  policies,  (e)  troop  advance  rate,  (f)  equipment  type,  and  (g)  troop  composition 
for  combined  campaign. 

In  the  context  of  ACAD  design,  the  decision  support  display  (DSD)  modeling 
represents  the  process  of  portraying  the  available  courses  of  action  information  on  a 
visual  space  so  as  to  enable  the  commander’s  ability  to  visualize  enfolding  decision 
events,  actions,  or  goals  in  real-time.  The  ACAD  database  has  command  and  control  (C2) 
behavioral  variables  such  as  explicit  intangibles  as  leadership  styles,  level  of  morale, 


communication,  and  shoreline  vulnerability.  Most  of  its  intelligence  activities  are  derived 
from  Table  of  Organizations  and  Equipment  (TOE),  which  break  down  a  combat  force 
into  its  constituent  units,  personnel,  and  weapons 

The  ACAD  has  three  major  sub  modules:  VIE,  CADIV  and  RDM.  First,  VIE 
(Visualizing  Intent  Environment)  is  a  constructive,  event-based  simulation  model  for 
predicting  and  updating  battle  states  based  on  the  input  from  a  specific  COA.  The  VIE 
model  uses  a  finite  state  automaton  to  model  the  Markovian  dynamics  of  the  battle 
events.  The  VIE’s  knowledge  base  is  based  on  heuristic  models  of  modified  Lanchester 
(Taylor  and  Parry,  1975)  and  Dupuy’s  (1985)  equations.  The  classical  Dupuy  [7] 
equations  use  long  time  spans  and  cannot  update  relative  force  ratio  and  effectiveness.  In 
our  model,  the  ACAD  can  calculate  COA  objective  function  values  in  time  increments  of 
one  hour  up  to  120  hours  (or  5  days)  of  continuous  battle. 

The  second  component  of  the  ACAD  is  the  Collective  Asset  Display  and  Intent 
Visualization  (CADIV)  model.  The  CADIV  model  has  three  decision  support  displays  in 
one  screen.  These  are  cognitive  display  of  the  commander’s  intent,  courses  of  action 
display,  and  performance  metrics  display.  The  CADIV  model  portrays  the  animated 
version  of  possible  avenues  of  approach  on  a  terrain  map  with  overlays.  The  cognitive 
display  of  the  commander’s  intent  portrays  the  space-time  dimension  of  the  battle 
objective  as  perceived  by  the  commander.  Changes  in  the  user’s  input  are  written  into  an 
Excel  database  and  displayed  graphically  based  on  the  simulated  state  of  events:  “delay 
operations”,  “defend”,  and  “counter-attack”.  An  avenue  of  approach  with  military 
symbologies  is  used  to  capture  the  mental  model  of  the  commander’s  intent  as  he/she 
conceptualizes  a  combat  strategy  designed  to  defeat  the  enemy. 

The  third  component,  Resource  Decision  Model  (RDM)  is  used  to  portray  the 
status  of  resources  and  their  impact  on  the  effectiveness  of  meeting  system  level 
objectives.  The  resource  management  display  is  designed  using  a  configural  display 
concept  (Barnes,  1995).  Changes  in  the  emergent  features  of  the  object  (e.g.,  shapes, 
patterns  and  proportions)  are  directly  related  to  various  system  states.  All  display 
modules  are  linked  to  each  other  through  data  templates  stored  in  a  common  database. 


The  ACAD  Methodology 

The  ACAD  software  considers  mobility  and  maneuver  issues  as  aggregate 
parameters  in  its  knowledge  base.  The  course  of  action  analysis  is  based  on  “Force-on- 
Force  Strength  Matching  (FOFSM)”.  Under  the  FOFSM  algorithm,  a  predefined  enemy 
force  structure  is  played  against  the  commander’s  (friendly)  selected  force  structure.  A 
selection  of  a  force  structure  defines  a  COA  with  battle  asset  configuration.  Each  COA 
outcome  is  detennined  by  the  FOFSM  algorithm  which  computes  several  measures  of 
performance,  including  effectiveness,  attrition  (for  personnel  and  weapons),  event  posture 
residency  time,  and  posture  risk  and  probabilities. 

The  ACAD  has  a  built-in  set  of  Tables  of  Organization  and  Equipment  (TOE)  for 
the  friendly  and  enemy  forces  for  each  level  of  the  unit  or  troop  composition.  Each  TOE 
defines  the  number  and  types  of  combat  units  to  be  involved  in  a  battle.  Selecting  a 
resource  block  in  the  ACAD’s  graphical  user  interface  (GUI)  automatically  references 


the  TOE  for  use  in  the  analytical  models.  The  ACAD  analytical  models  use  the  METT-T 
input  information  to  wargame  highly  abstracted  combats  situations. 

The  ACAD  domain  provides  the  use  some  options  to  select  the  desired  resource 
combinations,  force  strength  multipliers,  number  of  friendly  course  of  action  (FCOA), 
predefined  enemy  course  of  action  (ECOA),  planning  time  (maximum  of  120  hours), 
reserve  policy,  and  the  level  of  surprise.  At  the  end  of  a  wargame,  the  ACAD  displays  the 
relevant  measures  of  effectiveness  in  a  graphical  format.  The  user  can  visualize  this 
information  and  use  it  to  detennine  the  best  course  of  action.  The  ACAD  displays  the 
relevant  measures  of  effectiveness  in  a  graphical  format.  The  user  can  visualize  this 
information  and  uses  it  to  determine  the  best  course  of  action.  If  desired,  the  ACAD 
model  allows  the  user  to  override  the  computer  model’s  recommendation.  Exhibit  1 
shows  sample  user  input  screen  in  ACAD. 
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Exhibit  1 .  Sample  ACAD  input  screen. 

The  illustration  in  Exhibit  1  shows  a  sample  input  data  screen.  The  Editing  Tools 
allow  the  user  to  describe,  correct  and  use  widgets,  radio  buttons  and  icons.  There  is  a 
Guidance  facilitates  the  help  menu  and  provides  assistance  to  the  user.  The  User  Model  is 
selected  from  the  “Calibrate”  button.  The  first  window  to  the  left  allows  the  user  to  set 
simulation  parameters.  Upon  the  selection  of  the  either  the  “Blue  Force”  or  “Red  Force”, 
the  second  screen  on  the  right  hand  side  appears.  This  screen  allows  the  user  to  define  the 
asset  configurations  based  on  a  defined  RFR  desired.  The  marked  “X”  in  red  on  the  map 
area  defined  the  terrain  target  to  be  captured.  The  ACAD’s  user  interface  supports  direct 
manipulation  through  widgets,  windows  and  menus  (WWM).  The  WWM  design  is  based 
on  an  intelligent  object  (10)  metaphor  (Ntuen  and  Flanspal,  1999). 

The  first  feature  in  the  ACAD  model  is  the  subjective  information  of  the  military 
experts.  The  second  feature  is  the  heuristic  knowledge  base  developed  from  the  Colonel 
Dupuy’s  (1985)  combat  models  of  war.  Dupuy’s  model  was  chosen  because  of  two 
reasons.  First,  the  knowledge  base  contains  the  salient  characteristics  of  a  general 


battlefield.  Second,  previous  validation  studies  by  Dupuy  indicated  a  90  percent 
correlation  between  theoretical  and  actual  outcomes 

The  Dupuy ’s  combat  model  was  reviewed  and  found  to  fall  short  of  addressing 
modem  information-centric  warfare  environments.  Therefore,  the  VIE  model  was 
developed  in  ACAD  (Ntuen  and  Park,  2003,  in  this  volume)  to  recalibrate  Dupuy’s 
combat  parameters  to  modem  digital  weapons.  The  VIE’s  algorithm  converts  Dupuy’s 
operational  lethality  index  (OLI)  to  values  in  Battalion  equivalents  (BEs).  OLI  is  a 
measure  of  the  killing  power  of  weapons  ranging  from  the  sword  to  the  nuclear  bomb. 
Dupuy  provides  tables  of  values  as  well  as  formulas  for  calculating  OLIs  of  unlisted 
weapons  for  which  certain  characteristics  are  available.  The  OLI  terminology  is 
unfamiliar  to  many  commanders  who  are  more  accustomed  to  the  battalion  equivalent 
(BE)  concept.  DOD  (1997a)  and  DOD  (1977b)  have  information  for  both  American  and 
Russian  units  in  tenns  of  BEs. 


Sample  ACAD  Application 

The  ACAD  model  is  implemented  on  an  NT  Personal  Computer  using  C++  object 
program  language,  EXCEL™  spreadsheet,  and  Microsoft  VISUAL  BASIC  6.0.  Once  the 
user  is  in  the  ACAD  domain  (Exhibit  1),  the  user  follows  the  instructions  for  interaction 
and  user  input.  There  is  an  on-line  “HELP”  utility. 

To  demonstrate  the  application  of  the  ACAD,  consider  the  simple  case:  The  1st 
BDE  of  the  52nd  IN  Division  (MECH)  is  currently  in  a  defensive  posture  planning  for 
future  offensive  operations.  While  the  OPTEMPO  is  high,  enemy  contact  has  been  sparse 
for  the  last  48  hours  and  ability  to  conduct  unimpeded  planning  operations  has  given  the 
TOC  a  less  than  hectic  pace,  although  tempers  and  patience  are  on  edge  as  24  hour 
operations  have  taken  their  tolls  on  the  staff  officers  and  fatigue  and  stress  have 
compounded  the  difficulty  for  efficient  TOC  operations.  New  estimate  has  indicated  a 
potential  of  achieving  3:1  relative  force  ratio  (RFR)  because  of  new  supply.  The  current 
scheme  of  maneuver  that  is  in  the  BDE  commander’s  head  is  to  capture  terrain  “X”  by 
moving  from  south  flank  northwards.  The  problem  is  to  reconfigure  the  equipment, 
assess  the  troop  moral,  and  the  new  terrain,  and,  detennine  expected  attritions  for  friendly 
and  enemy  troops. 

In  this  problem,  we  can  start  the  simulation  by  selecting  120  hours,  which  is. 5 
days  of  continuous  battle  operation.  Exhibit  2  shows  the  original  input  data  (right  side) 
and  the  maneuver  animation  with  military  symbols  showing  the  units  selected  for  both 
friendly  force  (blue)  and  enemy  force  (red).  The  number  in  the  avenue  of  approach  arrow 
indicates  the  composite  troop  strength  at  the  end  of  the  simulation.  In  this  case,  the 
friendly  force  has  84%  strength  and  the  enemy  has  44%.  The  graph  below  indicates  time 
phase  plot  of  the  troop  strength.  The  user  can  recalibrate  the  input  and  run  ACAD  for 
different  asset  combinations.  Each  run  is  considered  to  be  equivalent  to  a  single  COA.  All 
output  data  under  each  COA  can  be  saved  in  a  separate  file  under  “MultipleAvenue” 
selection  bottom.  Specific  scenario  profiles  can  be  saved  and  used  for  a  re-run,  printed,  or 
imported  to  another  system  for  use.  For  example,  ACAD  attrition  data  is  used  by  Charles 
Analytic’s  FOX  model  as  a  front-end  input  data. 
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Exhibit  2  shows  the  graphical  plot  of  attrition  factors. 


Experimental  Evaluation  of  ACAD 

The  objective  of  the  study  was  to  detennine  the  level  of  usability  and  trust 
experienced  commanders  have  on  the  ACAD  as  a  battle  plan  assistant. 

Subjects:  We  were  concerned  that  task  experience  with  problem  familiarity  could  affect 
participant’s  trust  on  the  ACAD  software.  Such  influence  of  prior  knowledge  of  task  and 
use  of  decision  aids  has  been  illustrated  previously  by  several  studies  between  experts 
and  novices  (Benbasat  &  Taylor,  1978:  Vessey,  1985).  To  control  for  level  of  expertise, 
participants  with  no  command  experience  and  knowledge  of  COA  were  eliminated.  Six 
experienced  military  commanders  qualified  to  participate  in  the  study.  The  subjects’ 
composition  was  5  males  and  one  female  officer.  The  female  had  a  Lt.  Colonel  rank  in 
the  U.S.  Air  Force  and  was  a  commander  of  ROTC  Air  Force  Regiment  at  North 
Carolina  A&T  State  University  (NCA&T).  For  the  male  officers,  four  had  the  ranks  of 
Ft.  Colonel  and  two  with  Major  ranks  in  the  U.S.  Army.  Three  of  the  male  participants 
were  retired  (2  Ft.  Col,  1  Major);  the  Major  was  a  commander  of  a  reserve  army  unit  in 
Guilford  County,  North  Carolina.  Two  male  officers  were  active  duty  officers  (1  Ft.  Col 
was  the  commander  of  ROTC  Regiment  at  NCA&T,  and  1  Major  was  an  instructor  at 
NCA&T’s  ROTC  Regiment.  The  two  majors  had  unit  command  experience  during 
Korean  and  Desert  Storm  wars.  The  female  officer  had  command  experience  in  Kosovo, 
and  all  the  male  participants  with  the  rank  of  Ft.  Colonel  had  experience  commanding 
either  field  artillery  or  infantry  battalions.  All  participants  had  a  combined  military 
experience  of  1 13  man-years. 

Procedure:  Each  officer  participated  individually  in  the  study  during  the  months  of  June 
to  September  2000.  The  researcher  explained  the  general  procedure  of  the  study  and  then 
participants  read  and  signed  a  consent  form.  The  researcher  briefed  the  participants  on  the 


ACAD  and  the  types  of  input-output  of  the  software.  On  the  average,  all  participants 
were  trained  for  45  minutes  on  the  use  of  the  ACAD  software,  including  help  menus  and 
guidelines. 

Before  the  experiment,  the  participants  were  given  a  Decision  Aid  Expectation 
Form  (DAEF)  to  complete.  This  was  used  to  assess  individual  expectations  computer- 
based  decision  aids.  The  DAEF  had  subjective  rated  values  from  0  to  100,  with  “0” 
indicating  absolutely  not  important,  and  “100”  indicating  absolute  important.  The 
participant  must  select  their  scores  so  that  the  total  over  the  attributes  equal  to  100.  The 
aprior  expectations  were  later  used  as  weight  assignments.  Table  1  shows  the  contents  of 
DAEF. 

Table  1.  Sample  Decision  Aid  Expectation  Form 


Attribute 

Level  of  important  (0-100) 

Information  content/  management 

Reliability  of  decision 

Personal  dependency  of  decision  aids 

Robustness  of  decision  aid 

Confidence  on  decision  aids 

Total  score 

100 

The  problem  to  be  solved  was  given  in  the  “Sample  ACAD  Application”  section 


of  this  paper.  In  addition,  S2  (an  intelligent  scout)  has  determined  the  following 
information  about  the  enemy  surprise  strategy:  (a)  based  on  similar  encounter,  the 
probability  of  enemy  surprise  and  posture  is  completely  known;  (b)  the  probability  of 
surprise  and  location  is  50%  known;  (c)  there  is  information  on  the  enemy  surprise 
strategies.  These  conditions  are  known  as  “Low”,  “Medium”,  and  “High”  scenarios, 
respectively. 

In  the  first  experiment,  the  officers  were  told  to  manually  (pen-and  paper 
approach)  develop  their  COA  for  each  of  the  three  scenarios  using  the  given  asset 
information.  The  developed  solutions  were  collected  from  the  officers  by  the 
experimenter.  In  the  second  experiment,  the  officers  were  told  to  develop  their  COA 
solutions  using  the  ACAD  software  for  each  of  the  three  scenarios.  Each  participant  was 
given  sufficient  time  as  needed  to  complete  the  task.  Each  participant  perfonned  five 
trials  of  each  scenario  randomly  on  different  dates.  The  times  taken  to  complete  a  COA 
under  each  method  were  recorded  for  the  three  scenarios.  At  the  end  of  a  scenario,  each 
officer  was  asked  to  write  down  all  experiences  with  the  ACAD  software,  then  compete 
the  rating  of  the  software  using  the  DAEF  questionnaire  (See  Table  2).  At  the  end  of  the 
experiment,  each  participant  was  given  a  computer  print  out  of  all  the  six  scenarios — 
three  for  paper  and  pen  (manual)  and  three  with  automated  ACAD  support  (Low 
uncertainty,  Medium  uncertainty,  and  High  uncertainty). 

Table  2.  Sample  Decision  Aid  Rating _ 


Attribute 

Rate  your  perception  of  the  ACAD 
software  (0-100) 

Information  content/  management 

Reliability  of  decision 

Personal  dependency  of  decision  aids 

Robustness  of  decision  aid 

Confidence  on  decision  aids 

RESULTS 


1.  COA  Completion  Time:  Data  was  summarized  on  the  average  time  taken  by  the 
officers  to  derive  acceptable  COA.  This  is  shown  on  Table  3.  Figure  1  shows  the 
histogram  plot  of  the  distribution  of  time  in  Table  3. 


Table  3.  Means  and  standard  deviation  of  COA  times  (in  minutes) 


Experience 

level 

Low 

uncertainty 

COA 

Medium 

uncertainty 

COA 

High 

uncertainty 

COA 

Experts 

(Lt.Col.) 

2.18 

(std=0.26) 

3.51 

(std=0.62) 

3.937 

(std  =  0.51) 

Novices 

(Majors) 

3.94 

(std  =1.03) 

5.76 

(std  =  0.93) 

8.43 

(std  =  1.27) 

COA  Complexity 


Figure  1 .  Mean  time  distribution  for  COA  development 

A  2  (expertise  level)  X  3  (COA  complexity)  within-subjects  analysis  of  variance 
(ANOVA)  was  conducted  on  the  completion  times.  The  data  showed  that  there  were 
significant  differences  between  novices  and  experts  in  COA  completion  times,  F 
(1,24  =  249),  p  =  0.018.  The  levels  of  COA  uncertainty  was  significant,  F  (2,24)  = 
19.45,  p  =0.003.  There  was  no  interaction  between  the  levels  of  expertise  and  COA 
complexity,  F  (2,24)  =  19.45,  p  =  0.237.  Thus,  in  terms  of  COA  problem  solving, 
there  were  differences  between  novices  and  experts,  as  well  as  the  effects  of  surprise 
uncertainties  on  the  COA  development. 

To  test  the  effect  of  manual  and  decision  aid  on  COA  development,  A  2  (expertise 
level)  X  3  (COA  complexity)  x  2  (manual  vs.  ACAD)  within-subjects  analysis  of 
variance  (ANOVA)  was  conducted  on  the  completion  times.  The  data  showed  that 
there  were  significant  differences  between  the  use  of  the  ACAD  software  and  manual 
approach  to  COA  development  times,  F  (1,60  =  252.),  p  =  0.0003.  However,  when 
the  manual  and  the  ACAD  development  times  were  analyzed  by  COA  complexity, 
there  was  no  statistical  significant. 


2.  Perception  of  the  ACAD  software  as  a  COA  decision  aid:  Data  on  the  subjective 
rating  factors  and  the  officers’  trust  on  the  ACAD  tool  were  analyzed.  Table  4  gives  the 
average  values  for  each  factor.  Each  score  was  computed  by  multiplying  the  percentage 
equivalent  of  a  prior  weights  derived  from  Table  1  by  the  percentage  rating  values  derived 
from  Table  2.  Trust  values  were  computed  by  taking  the  average  of  all  composite  scores 
for  all  observations.  There  was  a  noticeable  significant  difference  on  the  officers’ 
perception  of  the  ACAD  software  as  a  COA  decision  aiding  tool,  (tcaiuiated  =  3.98  >  t  (io, 
0.025)  =  2.22  81),  p  =  0009.  Table  3  shows  the  average  trust  scores  for  a  COA  with  “Low” 
uncertainty. 


Table  4  Mean  perception  score  of  ACAD 


Attribute 

Expert  Score 

Novice  Score 

Information  content/ 
management 

0.72 

0.93 

Reliability  of  decision 

0.56 

0.87 

Personal  dependency  of 
decision  aids 

0.40 

0.58 

Robustness  of  decision  aid 

0.675 

0.90 

Confidence  on  decision  aids 

0.82 

0.85 

Trust  score 

0.913 

0.966 

3.  Expertise  and  COA  complexity:  At  the  5%  level  of  significance,  the  hypothesis  that 
there  is  some  strength  of  association  between  commanders  experience  and  COA  tasks 
could  not  be  rejected  (y2  (calculated)  =  3.21  <  %2  (2,o.95)=  5.99),  with  p  =  0.0034.  Additionally, 
the  student  T-statistics  (with  repeated  measures)  indicate  that  the  commanders  (Majors 
and  Lt.  Colonel)  used  in  the  study  did  show  some  differences  in  ACAD  trust  in  dealing 
with  uncertainties  (tcaiuiated  =  2.93  >  t  (5,0.05)  =  2.571,  p  =  0.0237.  Figure  2  shows  the  graph 
of  average  trust  scores  by  COA  scenario 


Figure  2.  Sample  plot  of  average  trust  scores  by  the  commanders 


Discussions,  Conclusions,  and  Lessons  Learned 


This  study  presents  the  development  of  the  ACAD  software  as  the  commander’s 
assistance.  Particularly,  the  commander’s  perception  of  the  ACAD  was  analyzed.  In 
tenns  of  COA  development  time,  the  experts  consistently  outperformed  novices  either 
with  the  use  of  the  ACAD  software  or  using  paper  and  pencil  (manual).  The  same  result 
was  true  for  the  times  to  develop  the  COA  based  on  their  complexities  as  defined  by 
uncertainty  of  enemy  surprises.  Also,  the  experts  show  some  differences  in  dealing  with 
uncertainties.  The  expert  group  was  able  to  discern  and  generate  low  risk  COAs  than  the 
majors.  In  terms  of  the  officers’  perception  of  the  ACAD,  there  was  a  significant 
difference  between  the  experts  and  novices.  The  experts  were  more  conservative  in  rating 
the  ACAD  software.  However,  they  showed  high  confidence  in  the  ACAD  (82%)  and 
agreed  that  the  ACAD  has  a  good  database  for  COA  planning  (72%).  The  reliability  and 
robustness  scores  were  low  (average  of  56%  and  67.5%).  For  the  novice  group,  trust  in 
the  ACAD  was  rated  highly  (average  score  of  96.6%).  All  other  factors  receive  high 
ratings  (Table  4)  except  their  perception  of  potential  dependency  on  the  decision  aid  in 
making  tactical  COA  decisions. 

Experimental  validation  of  the  ACAD  with  military  subjects  show  that  expert 
military  commanders  with  more  years  of  combat  experience  show  conservative  trust  on 
the  ACAD  as  a  planning  tool.  Commanders  with  less  experience  tend  to  show  more  trust 
and  dependency  on  the  ACAD.  The  reason  for  this  is  obvious.  Experts  who  use  decision 
aiding  systems  need  infonnation  that  matches  their  mental  models  as  well  as  situations 
that  pose  novel  solutions  (Adehnan,  1991).  In  the  ACAD  domain,  the  interface  design 
and  the  database  contain  information  already  familiar  to  the  expert  commanders.  The 
challenge  and  novel  part  of  the  ACAD  software  is  that  it  allows  the  commanders  to 
configure  military  assets  to  counter  a  given  relative  force  ratio;  and  they  can  war  game 
the  scenario  to  determine  the  attrition  factors  they  are  willing  to  accept.  For  the  novice, 
the  obvious  is  that  they  are  looking  for  any  solution  that  is  better  than  their  own 
(Benbasat  &  Taylor,  1978).  Therefore,  one  advantage  of  the  ACAD  software  is  that  it  can 
be  used  to  train  novices  in  COA  development  exercises.  The  major  drawback  in  the 
ACAD  is  that  the  intangible  variables  are  mostly  guesses.  This  is  not  true  in  the  real 
battlefield  scenarios 
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RATIONALE 

•TOO  MUCH  INFORMATION,  BUT  FIXED 
COGNITIVE  ASSET  OF  THE  COMMANDER 

•INFORMATION  PROCESSING  REQUIREMENT 
•PERCEIVING  &  RECEIVING 
•FILTERING  &  COMPRESSING 
•FUSING 

•MAKING  SENSE  OUT  OUT  OF  PROCESSED 
DATA  (JUDGMENT) 

•DECIDING 
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RATIONALE 

•  DYNAMIC  ENVIRONMENT: 

•CHANGES  IN  INFORMATION 
•INFORMATION  VOLUME  &  DENSITY 
•HETEROGENEITY  OF  INFORMATION 
•INFORMATION  SPEED 

•UNCERTAINTY: 

•INCOMPLETE  INFORMATION 
•UNAVAILABLE  INFORMATION 
•UNRELIABLE  SOURCE  OF  INFORMATION 
•FUZZINESS  IN  INFORMATION  DESCRIPTION 


SUPPORTING  COURSES  OF  ACTION  PLANNING  WITH  INTELLIGENT  MANAGEMENT  OF 

BATTLE  ASSETS 


RATIONALE 

•  BURDEN  OF  PROOF: 

•OPPORTUNITY  FOR  HUMAN  ERROR 
•COGNITIVE  WORKLOAD 
•TIME  &  DECISION  ACCURACY 

•SOLUTIONS: 

•COMPUTERIZED  DECISION  AIDS 
•Simulation  &  Modeling  Tool 
•Predict,  Forecast,  Estimate  Average  Policies 
•Used  for  Anticipation  &  Envisioning 
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SOME  EXISTING  DECISION  AIDS 

•  FOX-GA:  CO  A  MODEL  USING  GENETIC  ALGORITHM 
•CORAVEN:  INTELLIGENT  COLLECTION 

MANAGEMENT  USING  BAYESIAN  BELIEF  NET 
•OWL:  A  DECISION- ANALYTIC  WARGAMING  USING 
REAL-TIME  STATISTICAL  DATA  MINING 
•SCAT:  SENIOR  COMMANDER  AUTHORING  TOOL  USED 
FOR  PLAN  FRAMING 

•MODS AF :  3D  SIMULATION  WARGAME  WITH  METT-T 
•BVP:  BATTLE  PLANNING  &  VISUALIZATION  TOOL 
•There  are  many  other  tools  available* 

•Tools  are  task  dependent,  assumption-driven 
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ALTERNATIVE  COURSES  OF 
ACTION  DISPLAY  (ACAD) 


•COA  SIMULATION 
•DRIVERS  : 

•ANALYTICAL  MODELS 

•COL  DUPUY’S  COMBAT  MODEL 
(Refined) 

•BEHAVIORAL  MODELS 
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ALTERNATIVE  COURSES  OF 
ACTION  DISPLAY  (ACAD) 


•DRIVERS  : 

•TABLES  OF  ORGANIZATION  & 
EQUIPMENT 
•INTUITIVE  GUI 

•MEET-T  (MISSION,  ENEMY,  TROOPS, 
TERRAIN,  &  TIME) 

•Drilled-down,  granular  information  level 
•C2  Intangibles  (often  ignored  in  other  models) 
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ALTERNATIVE  COURSES  OF 
ACTION  DISPLAY  (ACAD) 


•OUTPUTS  : 

•Relative  force  ratio 
•Composite  attrition  factors 
•Troop  advance  rate 
•Time  base  performance  data 
•Graphical  displays 

•Battle  state  postures  (Attack,  defend,  etc.) 
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TOE 
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SAMPLE  APPLICATION 


*j 


Inputs 


Mission  |  Blue  Force  |  Red  Force  |  Environment  |  Assessments  | 


Number  of  hours 
to  model  (1  - 1 20) 


Friendly  reinforcement  |  q 
threshold  [X] 


h 


Minimum  friendly 
advance  (0-300  km) 


h 


2  Mission  [BUe  Force  ;)  Red  Force  |  Environment  |  Assessments  | 

Reserve  Unit  Posture  |HastyAttack  7] 

tt  of  Units  Unit  Types  X  Strength 

1 


(•  None 
f  Infantry 


0 


1] 


Maximum  acceptable  1  25  Minimum  enemy  1  5Q 

friendly  attrition  ( X )  '  attrition  (X)  ' 

I  I 

f'  Mechanized  1 0 

|M2  Bn  J 

|ioo 

f'  Armored  1 0 

|M1A1  Bn  _^J 

|ioo 

C  Atmd  Cavalry  [o  _▼] 

|Sqdn 

|ioo 

J  J 

C  Field  Artillery  |o  _^J 

|l55 MM  B^J 

|ioo  J 

Enemy  reinforcement  |  q 
threshold  [X) 


Rocket  Artillery  [o 
r  Air  Defense  h) 


h 


C  Attk  Helicopters  [o 
r  Ail  Cavalry  [o"" 


1 

1 


Bn 


MLRS  Bn 


ADA  Bty 


AH-64  Bn 


Sqdn  T] 


100 


3 


100 


1] 


100 


100 


1 

1 


100 


1 


OK 

Cancel 

Apply 

OK 

Cancel 

Apply 
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SAMPLE  APPLICATION 


i  <  I _ I 

6  hours  out  of  total  6  hours  ||T||||||||||||||||||||||||||||||||||||||||||||||||||||IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIV  Simulating  battle  6  hrs /  120  hrs... 
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SAMPLE  APPLICATION 


Untitled  -  ACAD 


File  Design  Inputs  Design  Environment  Iransfer  MultipleAvenue  Help 

fl-lfes  ♦|^|»|a|V| 


^jnjxj 


A  |  B  |  C  |  D  |  E 

F 

G 

H 

i 

j 

K 

L 

1 

ACAD 

2 

3 

4 

5 

6 

Hour 

Friendly 

Attrition 

Enemy 

Attrition 

Combat 

Power 

Advance 

7 

8 

<%> 

<%i 

Ratio 

(Km)  1 

9 

1 

3.76 

9.32 

2.39 

0.387 

Results  in  time-based 
information  on 

10 

2 

7  47 

18 

2.36 

0.767 

11 

3 

11.1 

26.2 

2.54 

1.2 

12 

4 

14.5 

34 

2.75 

1.69 

13 

5 

17.9 

41.2 

3 

2.25 

14 

6 

21.1 

48 

3.29 

2.91 

and  advance 

15 

7 

24,2 

54.2 

3.63 

3.69 

16 

17 

18 

y 

< 

19 

20 

22 

Selection  of  this 

23 

24 

25 

26 

27 

►  ►!  \  Results  Table  /  Settings  /  Attrition  /  Combat  Power  Ratio  /  Advance  /  J  <  | 


7  hours  out  of  total  7  hours 
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SAMPLE  APPLICATION 


Writing  ACAD  Output  to  remote  Machines 
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ACAD  EVALUATION 
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STUDY  PERIOD:  June  -September  2000 

DESIGN:  Within-Subject  study 

TRAINING:  MIN  (40  min.)  -  55  Min. 

PRIOR  INFORMATION:  Decision  Aid  Expectation  Form 
Questionnaire. 

TASK  COMPLEXITY :  Based  on  information  on  enemy  surprise 

*  Completely  Known  (LOW  COMPLEXITY) 

*  50%  Known  (MEDIUM  COMPLEXITY) 

*  No  Information  (HIGH  COMPLXITY) 
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EXPERIMENT  1 :  Paper  &  pen  (Manual)  (Option  of  5  trials) 

EXPERIMENT  2:  Decision  Aid  (Using  ACAD)  (Must  perform  5 
trials). 

TIME  LIMIT :  Sufficient,  open-ended  time  allowed. 

POST  TEST :  Rate  ACAD  on  given  trust  metric 

SCENARIO  PRESENTATION:  Random  on  different  days  and 
trials. 
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RESULTS 

COA  COMPLETION  TIMES 


DATA  ON  ACAD  ONLY 


Experience 

level 

Low 

uncertainty 

COA 

Medium 

uncertainty 

COA 

High 

uncertainty 

COA 

Experts 

(Lt.Col.) 

2.18 

(std=0.26) 

3.51 

(std=0.62) 

3.937 

(std  =  0.51) 

Novices 

(Majors) 

3.94 

(std  =1.03) 

5.76 

(std  =  0.93) 

8.43 

(std  =  1.27) 
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RESULTS 

COA  COMPLETION  TIMES 

DATA  ON  ACAD  ONLY 


SUPPORTING  COURSES  OF  ACTION  PLANNING  WITH  INTELLIGENT  MANAGEMENT  OF 

BATTLE  ASSETS 


RESULTS 

COA  COMPLETION  TIMES 

DATA  ON  ACAD  ONLY 

•2  (Expertise)  X  3  (COA  Complexity)  Within-subject  ANOVA: 
•Significant  differences  between  Cols.  &  Majors  (F  =  249;  p  =  0.018) 
•Information  Complexity  has  effect  on  task  times:  F  =  19.45,  p  =  0.003) 
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RESULTS 

COA  COMPLETION  TIMES 

DATA  ON  MANUAL  VS  ACAD 

•2  (Expertise)  X  3  (COA  Complexity)  X  2  (TookManual  vs  ACAD): 

•ACAD  VS.  MANUAL  Task:  F  =  252,  p  =  0.0003) 

•ACAD  Times  were  between  40-62%  less 

•ACAD  vs  MANUAL  &  Level  of  Complexity:  No  significant 
•Task  complexity  affected  COA  development  equally 
•May  be  due  to  preprocessing  time  required 
•Need  further  analysis 
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RESULTS 

ACAD  PERCEPTION  AS  A  COA  ASSISTANT:  LOW  COMPLEXITY 
(t  =  3.98,  p  =  0.0009:  Differences  between  Cols.  &  Majors) 


Attribute 

Expert  Score 

Novice  Score 

Information  content/ 
management 

0.72 

0.93 

Reliability  of 
decision 

0.56 

0.87 

Personal  dependency 
of  decision  aids 

0.40 

0.58 

Robustness  of 
decision  aid 

0.675 

0.90 

Confidence  on 
decision  aids 

0.82 

0.85 

Trust  score 

0.913 

0.966 
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RESULTS 

ACAD  PERCEPTION  AS  A  COA  ASSISTANT:  HIGH  COMPLEXITY 


Attribute 

Expert  Score 

Novice  Score 

Information  content/ 
management 

0.64 

0.83 

Reliability  of 
decision 

0.56 

0.87 

Personal  dependency 
of  decision  aids 

0.35 

0.65 

Robustness  of 
decision  aid 

0.62 

0.81 

Confidence  on 
decision  aids 

0.72 

0.89 

Trust  score 

0.82 

0.93 
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fix 


RESULTS 

Expertise  and  Task  Complexity  affect  trust  on  Decision  Aid 
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CONCLUSIONS 

•STUDY  SEEKED  TO  ANSWER: 

•Does  Decision  Aid  Help  Commanders  in  COA  Planning? 
•Do  Expertise  &  Task  Experience  Affect  Commander’s 
Perception  of  Decision  Aid  Trust? 

•FINDINGS: 

•1.  Decision  Aids  Support  COA: 

•Time  reduction  (observed) 

•Judgment  errors  (observed,  to  be  analyzed) 

•Real-time  asset  combination 
•Robust — ’’What  if’  &  “What  next” 

•Can  generate  multiple  CO  As 
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CONCLUSIONS 

•STUDY  SEEKED  TO  ANSWER: 

•Does  Decision  Aid  Help  Commanders  in  COA  Planning? 

•Do  Expertise  &  Task  Experience  Affect  Commander’s 
Perception  of  Decision  Aid  Trust? 

•FINDINGS: 

•2.  Commanders  with  COA  expertise  and  command  experience 
consistently  performed  better  than  those  with  less  expertise  & 
experience: 

•Dependency  on  mental  model 

•Comparative  judgment  (subject  ACAD  performance  to 
rigorous  field- value  judgment) 

•  Ssppk  for  information  not  known  or  if  known  nppH 
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CONCLUSIONS 

•STUDY  SEEKED  TO  ANSWER: 

•Does  Decision  Aid  Help  Commanders  in  COA  Planning? 
•Do  Expertise  &  Task  Experience  Affect  Commander’s 
Perception  of  Decision  Aid  Trust? 

•FINDINGS: 

•3.  Commanders  with  more  experience  tend  to  show 
conservative  trust  on  ACAD  while  those  with  less  command 
experience  tend  to  show  over  reliance  (more  trust): 

•Look  for  estimates  that  “make  sense” 

•Attrition  factor  watched  with  “passion”;  need  every  asset 
and  logistics  to  minimize  attrition. 
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CONCLUSIONS 

FURTHER  STUDY: 

•Meta  cognition  study  on  what  commanders  really  look  for  in 
in  computer  decision  aids.  It  is  not  sufficient  to  simulate.  How 
the  result  of  simulation  is  used  is  important.  May  help  to  reduce 
the  magnitude  and  scalability  of  simulation  models 

•More  “Intelligent  Interface”  for  the  novice  user.  Avoid  raising 
hopes.  Allow  the  user  to  use  “INQUIRY”  methods  within  the 
interface  to  “EXPLORE”  strategies. 

•Enhance  Fuzzy  data  fusion  in  ACAD. 

•Extend  war  game  board  to  multiple  objectives  or  area  of 
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